Scene-based people metering for audience measurement

ABSTRACT

Scene-based people metering for audience measurement is disclosed. An example method disclosed herein to perform people metering for audience measurement comprises segmenting image frames depicting a location in which an audience is expected to be present to form a sequence of scenes, for a first scene in the sequence of scenes, identifying a key frame representing a first sequence of image frames corresponding to the first scene, and processing the key frame to identify an audience depicted in the first scene.

FIELD OF THE DISCLOSURE

This disclosure relates generally to audience measurement and, moreparticularly, to scene-based people metering for audience measurement.

BACKGROUND

Audience measurement systems typically include one or more device metersto monitor the media presented by one or more media presentation deviceslocated at a monitored site. Many such audience measurement systems alsoinclude one or more people meters to obtain information characterizingthe composition(s) of the audience(s) in the vicinity of the mediapresentation device(s) being monitored. Prior people meters generallyfall into two categories, namely, active people meters or passive peoplemeters. An active people meter obtains audience information by activelyprompting an audience member to press an input key or otherwise enterinformation via the people meter. A passive people meter obtainsaudience information by passively monitoring the audience, usually byusing facial recognition techniques to count and/or identify theindividual audience members included in the audience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example audience measurement systememploying scene-based people metering as disclosed herein.

FIG. 2 is a block diagram of an example scene-based people meter thatcan be used to implement the example audience measurement system of FIG.1.

FIG. 3 is a block diagram of an example scene change detector that maybe used to implement the example scene-based people meter of FIG. 2.

FIG. 4A illustrates a first example operation of the scene changedetector of FIG. 3 to segment images into scenes.

FIG. 4B illustrates a second example operation of the scene changedetector of FIG. 3 to cluster scenes into scene clusters.

FIG. 5 is a block diagram of an example audience recognizer that may beused to implement the example scene-based people meter of FIG. 2.

FIG. 6 is a flowchart representative of example machine readableinstructions that may be executed to implement the example scene-basedpeople meter of FIG. 2.

FIG. 7 is a flowchart representative of example machine readableinstructions that may be executed to implement the example scene changedetector of FIG. 3.

FIG. 8 is a flowchart representative of first example machine readableinstructions that may be executed to implement the example audiencerecognizer of FIG. 5.

FIG. 9 is a flowchart representative of second example machine readableinstructions that may be executed to implement the example audiencerecognizer of FIG. 5.

FIG. 10 is a flowchart representative of third example machine readableinstructions that may be executed to implement the example audiencerecognizer of FIG. 5.

FIG. 11 is a block diagram of an example processing system that mayexecute the example machine readable instructions of FIGS. 6-8 and/or 9to implement the example scene-based people meter of FIG. 2, the scenechange detector of FIG. 3, the example audience recognizer of FIG. 5and/or the example audience measurement system of FIG. 1.

DETAILED DESCRIPTION

Example methods, apparatus and articles of manufacture to implementscene-based people metering for audience measurement are disclosedherein. As noted above, prior people meters for audience measurement aregenerally either active or passive. An active people meter obtainsaudience information by actively prompting an audience to enterinformation for audience member identification. A passive people meterobtains audience information passively, usually by capturing images ofthe audience using a camera and employing facial recognition to identifythe individual audience members included in the audience. As usedherein, facial recognition processing includes processing to one or bothof (i) detect (e.g., to count) faces in an image/scene and/or (ii)identify particular person(s) corresponding to particular face(s) in animage/scene. Similarly, audience recognition includesoperations/processing to (i) detect (e.g., to count) audience members inan image/scene and/or (ii) identify particular audience member(s) in animage/scene. Active people meters are generally simpler and less costlythan passive people meters, but are prone to measurement error due toaudience fatigue over time, lack of audience compliance, etc. Passivepeople meters do not rely on audience compliance and, thus, can be morereliable, but also require substantially more computing resources toimplement accurate facial recognition processing. As such, passivepeople meters are often too costly to deploy in a statisticallysignificant number of monitored sites. Additionally, the potentiallycostly facial recognition processing can slow down the image capturerate, making metering results less granular in time.

Scene-based people metering, as disclosed herein, employs passive peoplemetering, which can improve measurement accuracy and reduce reliance onaudience compliance relative to prior active people metering approaches.However, unlike prior passive people metering approaches, scene-basedpeople metering as disclosed herein further focuses audience recognitionprocessing on a subset of the captured image frames (e.g., the keyframes described below) corresponding to changes in the audienceenvironment being metered, then backtracks or otherwise propagates theresults of the recognition processing performed on these particularframes to the other captured image frames. Accordingly, in at least someexamples, scene-based people metering as disclosed herein does not incurthe costs associated with prior passive metering techniques that requirefacial recognition to be performed on each captured image of theaudience.

Example methods disclosed herein to perform scene-based people meteringfor audience measurement include obtaining image frames depicting alocation in which an audience is expected to be present. The examplemethods also include segmenting the image frames to form a sequence ofscenes. The example methods further include, for a next scene in thesequence of scenes, processing a key frame representing a first sequenceof image frames forming the next scene to identify an audience depictedin the next scene. For example, such processing of the key frame caninclude performing facial recognition processing on the key frame toidentify audience members depicted in the next scene. Some examplemethods perform facial recognition processing on the key frame with theassistance of previously determined identification information. Someexample methods rank the scenes (or the key frames representing thescenes) and process the key frames based on the rankings.

In some disclosed examples, the captured image frames are segmented toform the sequence of scenes by determining image signaturesrepresentative of the image frames, and then including successive frameshaving similar image signatures in respective sequences of image frames(also referred to herein as segments of image frames). In such examples,each sequence (segment) of image frames forms a respective scene in thesequence of scenes. In some examples, the key frame used to representthe next scene can, therefore, correspond to at least one of a startingimage frame, a midpoint image frame, an ending video image or astatistical combination of the sequence of image frames forming the nextscene.

Some disclosed example methods process the key frame used to representthe first sequence of image frames forming the next scene by comparingan image signature representative of the key frame with a set ofreference signatures representative of a set of reference scenes todetermine a comparison result. In such examples, the key frame is thenprocessed based on the comparison result to identify audience membersdepicted in the next scene. For example, the key frame can be assignedto a processing queue with a first priority (e.g., and with nosuggestion for performing audience recognition) when the comparisonresult indicates that the image signature does not match at least onereference signature in the set of reference signatures. However, the keyframe can be assigned to the processing queue with a second priority,which is lower than the first priority, (and with a suggestion forperforming audience recognition) when the comparison results indicatesthat the image signature matches at least one reference signature in theset of reference signatures. Such example methods then use thepriorities associated with the key frames included in the processingqueue to determine how to prioritize performing facial recognitionprocessing on the different key frames in the queue to identify theaudience members depicted in the respective scenes represented by thekey frames. Such example methods can also use any suggestions forperforming audience recognition that are provided with the key frame(s).For example, such a suggestion for a particular key frame can includelikely facial positions, sizes, orientations, etc., that are associatedwith a reference signature that was determined to match the imagesignature representative of the key frame. Because key frames are storedin the processing queue, audience recognition processing can beperformed offline (e.g., not in real time), which can decouple theeffects of such processing on the image capture rate and the associatedtime granularity of people metering for audience measurement.

Additionally or alternatively, some disclosed example methods determinewhether the comparison result indicates that a key image signature,which is representative of the key image for the next scene, matches afirst reference signature in the set of reference signatures. In suchexamples, audience identification information already associated with afirst reference scene represented by the first reference signature isused to infer or otherwise identify the audience members depicted in thenext scene. In some examples, the audience identification informationalready associated with the matching first reference scene is reportedin lieu of performing audience recognition on the key image for the nextscene if such audience recognition processing has not completed by thetime the reporting is to occur. Additionally or alternatively, someexamples methods associate audience identification information with thekey image signature, which is representative of the key image for thenext scene. Such example methods then include the key image signature inthe set of reference signatures for comparison with a second imagesignature representative of a subsequent scene occurring after the nextscene.

Some disclosed example methods further cluster the sequence of scenesinto a set of one or more distinct scenes (also referred to herein asunique scenes, representative scenes, etc.). In such examples, thesequences of scenes are clustered into clusters of matching scenes thathave matching key image frames and/or matching key image signatures (orsubstantially matching key image frames/signatures within a tolerancelevel, threshold, etc.). Each scene cluster is represented by arepresentative key frame selected from other determining by combiningthe key frames of the scenes included in the cluster. In examplesemploying scene clustering, audience recognition processing can beperformed on just the representative key frame for each cluster (e.g.,by storing just the representative key frames for the clusters in theprocessing queue described above), with the audience identificationresults for the representative key of a particular scene cluster beingpropagated (e.g., back-tracked, assigned, etc.) to all the scenesincluded in that cluster. Furthermore, in some examples, the sceneclusters can be ranked based on, for example, the size of the clusters,where the size of a cluster corresponds to the number of images includedin the cluster (e.g., corresponding to the total number of images overthe image sequences forming the scenes included in the cluster). In suchexamples, the representative key frames for the scene clusters stored inthe processing queue can be assigned ranks based on their cluster sizes,and the (potentially costly) audience recognition processing can beapplied to the representative key frames in accordance with the assignedranks (e.g., with larger clusters being processed before smallerclusters).

Example apparatus to implement scene-based people metering for audiencemeasurement, and example articles of manufacture (e.g., storage media)storing machine readable instructions which, when executed, causeexample machine(s) to perform scene-based people metering for audiencemeasurement, are also disclosed herein.

Turning to the figures, a block diagram of an example audience meteringsystem 100 employing scene-based people metering as disclosed herein isillustrated in FIG. 1. The example audience measurement system 100supports monitoring of media content exposure to audiences at one ormore monitored sites, such as the example monitored site 105 illustratedin FIG. 1. The monitored site 105 includes an example media presentationdevice 110 and an example audience area 115. The audience area 115corresponds to one or more locations at the monitored site 105 in whichan audience 120 is expected to be present when consuming media content(e.g., viewing and/or hearing the media content, interacting with thecontent, etc.) presented by the media presentation device 110. Theaudience area 115 can include, but is not limited to, a room containingthe media presentation device 110, a sitting area in front of the mediapresentation device 110, etc. Although the example of FIG. 1 illustratesone monitored site 105, scene-based people metering as disclosed hereincan be used in audience measurement systems 100 supporting any number ofmonitored sites 105.

The audience measurement system 100 of the illustrated example includesan example site meter 125, also referred to as a site unit 125, a homeunit 125, etc., to monitor media content presented by the mediapresentation device 110. To support scene-based people metering at themonitored site 105 in accordance with the examples described herein, theexample audience measurement system 100 of FIG. 1 also includes anexample scene-based people meter 130, which is described in greaterdetail below. In the illustrated example, the site meter 125 determinesaudience measurement data characterizing media content exposure at themonitored site 105 by combining metering data (also referred to ascontent metering data, content monitoring data, content measurementdata, tuning data, etc.), which is determined by monitoring the mediapresentation device 110, with audience identification data (alsoreferred to as demographic data, people meter data, etc.), which isprovided by the scene-based people meter 130. The audience measurementmeter 125 then stores and reports this audience measurement data via anexample network 135 to an example data processing facility 140. The dataprocessing facility 140 performs any appropriate post-processing of theaudience measurement data to, for example, determine audience ratingsinformation, identify targeted advertising to be provided to themonitored site 105, etc. In the illustrated example, the network 135 cancorrespond to any type(s) and/or number of wired and/or wireless datanetworks, or any combination thereof.

In the illustrated example, the media presentation device 110 monitoredby the site meter 125 can correspond to any type of audio, video and/ormultimedia presentation device capable of presenting media contentaudibly and/or visually. For example, the media presentation device 110can correspond to a television and/or display device that supports theNational Television Standards Committee (NTSC) standard, the PhaseAlternating Line (PAL) standard, the Systeme Électronique pour Couleuravec Mémoire (SECAM) standard, a standard developed by the AdvancedTelevision Systems Committee (ATSC), such as high definition television(HDTV), a standard developed by the Digital Video Broadcasting (DVB)Project, etc. As another example, the media presentation device 110 cancorrespond to a multimedia computer system, a personal digitalassistant, a cellular/mobile smartphone, a radio, etc.

The site meter 125 included in the audience measurement system 100 ofthe illustrated example can correspond to any type of metering devicecapable of monitoring media content presented by the media presentationdevice 110. As such, the site meter 125 may utilize invasive monitoringinvolving one or more physical connections to the media presentationdevice 110, and/or non-invasive monitoring not involving any physicalconnection to the media presentation device 110. For example, the sitemeter 125 may process audio signals obtained from the media presentationdevice 110 via a microphone and/or a direct cable connection to detectcontent and/or source identifying audio codes and/or audio watermarksembedded in audio portion(s) of the media content presented by the mediapresentation device 110. Additionally or alternatively, the site meter125 may process video signals obtained from the media presentationdevice 110 via a camera and/or a direct cable connection to detectcontent and/or source identifying video codes and/or video watermarksembedded in video portion(s) of the media content presented by the mediapresentation device 110. Additionally or alternatively, the site meter125 may process the aforementioned audio signals and/or video signals togenerate respective audio and/or video signatures from the media contentpresented by the media presentation device 110, which can be compared toreference signatures to perform source and/or content identification.Any other type(s) and/or number of media content monitoring techniquescan additionally or alternatively be supported by the site meter 125.

In the example of FIG. 1, the audience measurement system 100 includesthe example scene-based people meter 130 to capture information aboutthe audience 120 that is exposed to the media content presented by themedia presentation device 110. A block diagram of an exampleimplementation of the scene-based people meter 130 of FIG. 1 isillustrated in FIG. 2. The example scene-based people meter 130 of FIG.2 includes an example imaging sensor 205, such as a camera, from whichimage frames are obtained that depict a scene in which the audience 120is expected to be present. For example, the imaging sensor 205 of thescene-based people meter 130 can be positioned such that its field ofview 210 includes the audience area 115. As described in greater detailbelow, the scene-based people meter 130 uses the captured image framesto identify the audience 120 passively or, in other words, withoutrequiring the audience 120 to actively provide audience identificationinformation (e.g., demographic information).

To control operation of the imaging sensor 205, the scene-based peoplemeter 130 of FIG. 2 includes an example sensor interface 215. The sensorinterface 215 can be implemented using any appropriate type(s) and/ornumber of interface(s) to enable controlling when the imaging sensor 205is to capture image frames, to enable receiving the captured imageframes, and to enable storing the captured image frames. In theillustrated example, the sensor interface 215 causes the imaging sensor205 to capture image frames at a frame rate that may be pre-configured,specified as a parameter during configuration of the scene-based peoplemeter 130, etc. The sensor interface 215 also causes the captured imageframes to be stored in an example image storage 220 using anyappropriate image format. The image storage 220 may be implemented byany type of a storage or memory device, a database, etc., such as themass storage device 1128 and/or the volatile memory 1114 included in theexample processing system 1100 of FIG. 11, which is described in greaterdetail below.

The example scene-based people meter 120 of FIG. 2 further includes anexample scene change detector 225 to segment the image frames capturedby the image sensor 205 and stored in the image storage 220 into asequence of scenes for which audience identification is to be performed.A scene corresponds to a sequence (or segment) of successive imageframes having substantially similar image characteristics. For example,a scene may correspond to a sequence of successive images frames havingforeground components (e.g., which may correspond to members of theaudience 120) and a background (e.g., which may correspond to theenvironment of the audience area 115) that are relatively unchangingover time. The scene change detector 225 of the illustrated exampleutilizes scene change detection to mark the beginning image frame andthe ending image frame corresponding to a scene, thereby grouping thesequence of image frames from the beginning image frame to the endingimage frame into the scene. In some examples, the scene change detector225 performs scene change detection by creating an image signature foreach frame in the sequence of captured image frames (possibly aftersubsampling) and then comparing the image signatures to determine when ascene change occurs. For example, the scene change detector 225 maycompare the image signature corresponding to the starting image of ascene to the image signatures for one or more successive image framesfollowing the starting frame. If the image signature for the startingframe does not differ significantly from a successive frame's imagesignature (e.g., such as when the audience 120 has not changed betweenthe two image frames), the successive frame is included in the sequenceforming the same scene as the starting frame. However, if the imagesignatures are found to differ significantly (e.g., such as when theaudience 120 has changed between the two image frames), the successiveframe that differs is determined to be the start of a new scene andbecomes the first frame in the sequence forming that new scene.

After a scene is detected, the scene change detector 225 of theillustrated example also determines key frame(s) and an image signatureof the key frame(s) (referred to herein as the key image signature) thatare to be representative of the scene. The scene change detector 225stores the key frame(s) and the key image signature(s) in the imagestore 220. For example, the key frame and the key image signature forthe scene may be chosen to be the frame and signature corresponding tothe first frame in the scene, the last frame in the scene, the midpointframe in the scene, etc. In other examples, the key frame and the keyimage signature may be determined to be an average and/or some otherstatistical combination of the sequence of image frames and/orsignatures corresponding to the detected scene. In some examples, suchas in the case of scenes having long image sequences, multiple keyframes and associated key image signatures may be used to represent thescene (e.g., by choosing frames interspersed within the image sequenceforming the scene, such as by choosing every Nth image frame to be a keyframe, where N is some positive integer number). By segmenting the imageframes captured by the imaging sensor 205 into scenes corresponding toimage sequences represented by key frames, the scene-based people meter130 can focus audience recognition processing on just the key framescorresponding to the scene changes, unlike prior passive people metersthat performing audience recognition on all captured image frames.

In some examples, because the scene change detector 225 processes theimage frames after they are stored in the image storage 215, the scenechange detector 225 can perform its scene detection processing off-line,rather than in real-time as the image frames are captured. Such off-lineprocessing can involve few processing resources that would be requiredfor real-time processing, thereby further reducing the cost of thescene-based people meter 130 relative to prior passive people meters. Anexample implementation of the scene change detector 225 is illustratedin FIG. 3, which is described in greater detail below. In some examples,the scene change detector 225 performs scene change detection inreal-time, whereas the subsequent audience recognition processing of thescenes (e.g., discussed below) is performed off-line on the key imagesrepresentative of the scenes.

To perform audience recognition and identification, the examplescene-based people meter 130 of FIG. 2 includes an example audiencerecognizer 230. In the illustrated example, the audience recognizer 230retrieves, from the image storage 220, the key frames corresponding tothe scenes detected by the scene change detector 225. The audiencerecognizer 230 then performs facial recognition processing, and/or anyother number and/or type(s) of image processing, on each key frame toidentify member(s) of the respective audience 120 depicted in each oneof the key frames.

In some examples, when processing a key frame corresponding to adetected scene, the audience recognizer 230 additionally oralternatively compares the key frame to a set of reference images storedin an example reference storage 235. (As discussed in greater detailbelow, such comparison of key frame(s) to reference images can beperformed by comparing key image signature(s) representative of the keyframe(s) with reference signatures representative of the referenceimages. In such examples, the reference signatures may be stored in thereference storage 235 instead of, or in addition to, the referenceimages.) The reference storage 235 may be implemented by any type of astorage or memory device, a database, etc., such as the mass storagedevice 1128 and/or the volatile memory 1114 included in the exampleprocessing system 1100 of FIG. 11, which is described in greater detailbelow. The reference images (and/or reference signatures) stored in thereference storage 235 are representative of a history of referencescenes for which audience recognition has been performed by thescene-based people meter 130 in the past. If the key frame for thedetected scene being processed matches at least one of the referenceimages, the audience recognizer 230 determines that the audience 120depicted in the key frame corresponds to the reference audienceassociated with the matching reference image (or signature). In suchcases, the audience recognizer 230 can use reference audienceidentification information (e.g., reference demographic information)already associated with the matching reference audience to identify theaudience 120 depicted in the key frame of the detected scene beingprocessed. Furthermore, when the key frame for the detected scene beingprocessed matches at least one of the reference images, the audiencerecognizer 230 may, in some example, also perform facial recognitionprocessing, and/or any other number and/or type(s) of image processingon the key frame to verify/update the reference audience identificationinformation already associated with the matching reference image.However, the audience recognizer 230 may prioritize such processing onthe key frame at a lower priority than if the key frame had not matchedany of the reference images and, thus, had corresponded to a newaudience to be identified.

After identifying the audience 120 depicted in the key frame of thescene being processed (e.g., via facial recognition processing, etc.,and/or by matching the key frame with a reference image), the audiencerecognizer 230 reports the audience identification information for theaudience 120 to the site meter 125 (not shown in FIG. 2). In someexamples, the audience recognizer 230 stores the audience identificationinformation with the key frame (or key image signature) in the referencestorage 235 as another reference image (or signature) and associatedreference audience identification information for comparing with asubsequent key frame representative of a subsequent scene to beprocessed. An example implementation of the audience recognizer 230 isillustrated in FIG. 5, which is described in greater detail below.

An example implementation of the scene change detector 225 of FIG. 2 isillustrated in FIG. 3. The example scene change detector 225 of FIG. 3employs image signatures and image signature comparison to segment theimage frames stored in the image store 220 into a sequence of scenescorresponding to detected changes in the audience area 115. Generally,an image signature is a proxy representative of the associated image,and can take the form of, for example, one or more digital values, awaveform, etc. Because image signatures are proxies representing theirassociated images, the signatures of two images can be compared todetermine whether their respective images are substantially similar oridentical. Generally, if two image signatures match or substantiallymatch (e.g., at least within some tolerance or deviation level), thenthe respective images they represent are substantially similar oridentical. Typically, signature comparison is simpler and requires lessprocessing resources than direct image comparison, and is more robustagainst insignificant changes between the images. Moreover, usingsignatures to compare images can address privacy concerns by removingthe need to store images of scenes potentially depicting audiencemembers.

Thus, to implement image grouping for scene detection, the scene changedetector 225 of FIG. 3 includes an example signature generator 305, anexample signature comparator 310 and an example image segmenter 315. Thesignature generator 305 of the illustrated example generates imagesignatures representative of the image frames retrieved from the imagestorage 220 (e.g., and which were obtained by the imaging sensor 205).In some examples, the image signature generated by the signaturegenerator 305 for a given image frame corresponds to an image histogramof the luminance and/or chrominance values included in the image frame.Further examples of image signatures that can be generated by thesignature generator 330 include, but are not limited to, the examplesdescribed in U.S. Patent Publication No. 2011/0243459, entitled “Methodsand Apparatus to Detect Differences Between Images” and published onOct. 6, 2011; U.S. Patent Publication No. 2009/0123025, entitled“Methods and Apparatus to Measure Brand Exposure in Media Stream” andpublished on May 14, 2009; U.S. Patent Publication No. 2008/0068622,entitled “Methods and Apparatus to Identify Images in PrintAdvertisements” and published on Mar. 20, 2008; U.S. Publication No.2006/0153296, entitled “Digital Video Signature Apparatus and Methodsfor Use with Video Program Identification Systems” and published on Jul.13, 2006; U.S. Pat. No. 6,633,651, entitled “Method and Apparatus forRecognizing Video Sequences” and issued on Oct. 14, 2003; and U.S. Pat.No. 6,577,346, entitled “Recognizing a Pattern in a Video Segment toIdentify the Video Segment” and issued on Jun. 10, 2003. U.S. PatentPublication Nos. 2011/0243459, 2009/0123025, 2008/0068622 and2006/0153296, and U.S. Pat. Nos. 6,633,651 and 6,577,346, are herebyincorporated by reference in their respective entireties.

The example signature comparator 310 included in the scene changedetector 225 of FIG. 3 compares the image signatures generated by thesignature generator 305 for pairs of image frames stored in the imagestorage 220 (e.g., and which were obtained by the imaging sensor 205).The signature comparator 310 can implement any type(s) and/or number ofcomparison criteria, such as a cross-correlation value, a Hammingdistance, etc., to determine whether the image signatures for a pair ofimage frames match or substantially match within a particular tolerancelevel (e.g., which may be predetermined, specified as a configurationparameter or input, etc.).

The example image segmenter 315 included in the scene change detector225 of FIG. 3 uses the results of the image signature comparisonsperformed by the signature comparator 310 to segment the image framesstored in the image storage 220 (e.g., and which were obtained by theimaging sensor 205) into image sequences (also referred to as imagesegments) forming a respective sequence of scenes (e.g., where eachscene depicts a change in the audience 120 being monitored). The imagesegmenter 315 of the illustrated example segments image frames intoscenes by using the signature comparator 310 to compare image frames toa starting frame representative of a current scene being detected. Asdiscussed above, successive image frames that have similar imagesignatures are grouped together to form an image sequence representativeof a scene. In the illustrated example, to segment image frames into acurrent scene that begins with a starting image frame, the imagesegmenter 315 compares the image signature for a next image frame withthe image signature for the starting frame of the current scenecurrently being detected. If the signature comparator 310 determinesthat the image signature for the next image frame matches (orsubstantially matches within a particular tolerance level) the startingframe's image signature, the image segmenter 315 includes the currentimage frame in the sequence of images forming the current scene beingdetected. This procedure is then repeated for successive image framesuntil the signature comparator 310 determines that the image signaturefor one of the successive images does not match the starting frame'simage signature.

When the current image frame being processed is determined to not matchthe starting frame of the current scene being detected, the imagesegmenter 315 identifies this current image frame as starting a newscene. The image segmenter 315 also denotes the current image frame asthe starting frame for that new scene, and stores the image signaturefor the current frame for use as the starting image signature for thatnew scene. The example image segmenter 315 also marks the immediatelyprevious frame as the ending frame for the sequence of image framesforming the current scene that was just detected. In this way, thesequence of successive image frames beginning from a scene's startingimage frame and ending with the scene's ending image frame are grouped(or, in other words, segmented) to form the scene.

The scene change detector 225 of FIG. 3 further includes an example keyframe identifier 320 to determine the key frames used to represent thesequence of scenes formed by segmenting the image frames stored in theimage storage 220 (e.g., and which were obtained by the imaging sensor205) into image sequences, and to also determine the key imagesignatures representative of the key frames. In some examples, the keyframe identifier 320 determines the key frame and key image signaturefor a given scene to be the image frame and image signaturecorresponding to either the starting frame in the sequence of framesforming the scene, or the last frame in the sequence of frames formingthe scene, or the midpoint frame in the sequence of frames forming thescene, or any other frame in the sequence of frames forming the scene.In some examples, the key frame identifier 320 determines the key framefor a given scene to be an average or some other statistical combinationof the image frames included in the sequence of frames forming thescene. In such examples, the key frame identifier 320 may use thesignature generator 305 to generate the key image signature from theresulting key frame, or the key frame identifier 320 may perform thesame averaging or other statistical processing on the image signaturesfor the sequence of image frames forming the scene to determine aresulting key image signature. In some examples, to accurately identifya particular scene in a sequence of scenes, the key frame identifier 320also associates a scene's starting frame time/number and the scene'sending frame time/number with the key frame and/or key image signaturerepresentative of the scene.

FIG. 4A illustrates an example operation 400 of the scene changedetector 225 of FIG. 3 to segment image frames to form image sequencescorresponding to a respective sequence of scenes. In the exampleoperation 400 of FIG. 4A, the scene change detector 225 retrieves imageframes 405 from the image storage 220. As described above, the imageframes 405 are captured by the imaging sensor 205 and depict theaudience area 115 in which an audience 120 is expected to be present. Inthe example operation 400 of FIG. 4A, the scene change detector 225 thensegments the image frames 405 into an example sequence of scenes 410represented by respective example key frames 415. In the illustratedexample operation 400 of FIG. 4A, the scene change detector 225determines the key frames 415 to be the starting frames of each of theimage sequences forming the scenes 410. As can be seen from the exampleof FIG. 4A, focusing audience recognition processing on just the keyframes 415 instead of all of the image frames 405 can significantlyreduce the processing costs associated with the scene-based people meter130.

FIG. 4B illustrates an example operation 450 of the scene changedetector 225 of FIG. 3 to cluster scenes into scene clusters. Withreference to FIG. 3, in some examples, the scene change detector 225further includes an example image clusterer 325 to cluster the sequenceof scenes and associated key frames determined by the image segmenter315 and the key frame identifier 320 into clusters of matching scenes,where each scene cluster is represented by a representative key imageformed or selected from the key frames of the scenes included in thecluster. For example, the image clusterer 325 can compare the key imagesignatures for the key frames of the scenes determines by the scenechange detector 225 to identify matching key frame signatures (or keyframe signatures that substantially match with a tolerance level,threshold, etc.). In such examples, the scenes having matching key framesignatures are further clustered (e.g., grouped) to form a scenecluster. The representative key frame for the scene cluster can beselected from among the key frames of the scenes included in the scenecluster, or determined to be a combination (e.g., statisticalcombination) of the key frames of the scenes included in the scenecluster.

In some examples in which scene clustering is employed, audiencerecognition processing can be performed on just the representative keyframe for each cluster (e.g., by storing just the representative keyframes for the clusters in the processing queue 515 described in greaterdetail below), with the audience identification results for therepresentative key of a particular scene cluster being propagated (e.g.,back-tracked, assigned, etc.) to all the scenes included in thatcluster. Furthermore, in some examples, the scene clusters can be rankedbased on, for example, the size of the clusters. The size of a clustercorresponds to the number of images included in the cluster (e.g.,corresponding to the total number of images over all of the imagesequences forming all of the scenes included in the cluster). In suchexamples, the audience recognition processing can be applied to therepresentative key frames of the scene clusters in accordance with theranks assigned to the different scene clusters.

Turning to the example operation 450 of FIG. 4B, the image clusterer 325compares the key image signatures for the key frames 415 of the sequenceof scenes 410 to determine a scene clusters represented by therepresentative key frames 455, 460, 465 and 470. In the illustratedexample of FIG. 4B, the image clusterer 325 groups the first, third andseventh scenes into a first image cluster represented by therepresentative key frame 455. The image clusterer 325 also groups thesecond and fifth scenes into a second image cluster represented by therepresentative key frame 460. The image clusterer 325 further groups thefourth and eighth scenes into a third image cluster represented by therepresentative key frame 465. The image clusterer 325 finally groups thesixth scene into a third image cluster represented by the representativekey frame 470.

In the illustrated example of FIG. 4B, the representative key frames455, 460, 465 and 470 are then subjected to audience recognitionaccording to their rank, with the representative key frame 455 beingranked the highest because it represents the scene cluster having thelargest size, and the representative key frame 470 being ranked thelowest because it represents the scene cluster having the smallest size.After audience recognition processing is performed for a representativekey frame, the audience identification results are propagated to allscenes in the scene cluster corresponding to the representative keyframe. For example, the audience identification results from processingthe representative key frame 455 are propagated (e.g., backtracked orassigned to) the first, third and seventh scenes included in the firstimage cluster represented by the representative key frame 455.

An example implementation of the audience recognizer 230 of FIG. 2 isillustrated in FIG. 5. The example audience recognizer 230 of FIG. 5determines audience identification information (e.g., audiencedemographic data) for audiences 120 depicted in the key framesrepresenting the sequence of scenes detected by the scene changedetector 225. As described above, each scene corresponds to a likelychange in the audience 120 or the environment. In some examples, theaudience recognizer 230 of FIG. 5 varies the audience recognitionprocessing performed on the key frames representing the sequence ofscenes depending on whether a particular key frame (or signature)matches a reference image (or reference signature) in a set of referenceimages (or reference signatures) corresponding to reference audience(s)that have been previously identified by the audience recognizer 230. Insuch examples, the audience recognizer 230 of FIG. 5 can utilize imagesignature comparison to determine whether a particular key framesignature matches at least one of the reference signatures correspondingto the reference audience(s).

For example, the audience recognizer 230 of FIG. 5 includes an examplesignature comparator 505, which may be the same as or similar to thesignature comparator 310, to compare the key image signatures stored inthe image storage 220, and which are representative of the key framescorresponding to the detected scenes, with reference signatures (alsoreferred to as reference image signatures) stored in the referencestorage 235, and which are representative of the reference audience(s)that have been previously identified by the audience recognizer 230. Tocompare a particular key frame with the set of reference images, thesignature comparator 505 retrieves the key image signaturerepresentative of the key frame from the image storage 220. Thesignature comparator 505 then compares the key image signature withreference signatures retrieved from the reference storage 235 todetermine whether the key image signature and one of the referencesignatures match or substantially match within a particular tolerancelevel (e.g., which may be predetermined, specified as a configurationparameter or input, etc.). The signature comparator 310 can implementany type(s) and/or number of comparison criteria, such as across-correlation value, a Hamming distance, etc., to perform thesignature comparison.

The audience recognizer 230 of FIG. 5 also includes an example audienceidentifier 510 that performs audience recognition processing on aparticular key frame based on the comparison result determined by thesignature comparator 505 for the key frame. For example, if thecomparison result determined by the signature comparator 505 for aparticular key frame indicates that the key frame's image signature doesnot match any of the reference signatures, then the audience identifier510 can perform audience recognition processing, such as facialrecognition processing, on the key frame to identify the audience 120depicted in the scene represented by the key frame. However, if thecomparison result determined by the signature comparator 505 for theparticular key frame indicates that the key frame's image signaturematches (or substantially matches) one of the reference signatures, thenthe audience identifier 510 can forego audience recognition processingand, instead, use the reference audience identification informationalready associated with the reference scene represented by the matchingreference signature to identify the audience 120 depicted in thedetected scene represented by the key frame being analyzed. In suchexamples, the reference audience identification information (e.g.,reference demographic information) for the matching reference scene isretrieved from the reference storage 235 by the audience identifier 510and associated with the key frame currently being analyzed by theaudience identifier 510.

In some examples, the audience recognizer 230 of FIG. 5 additionally oralternative prioritizes the audience recognition processing performed ona particular key frame based on the comparison result determined by thesignature comparator 505 for the key frame. For example, the audiencerecognizer 230 of FIG. 5 can include an example queue 515 to store keyframes awaiting processing by the audience identifier 510. The queue 515may be implemented by any type of a storage or memory device, adatabase, etc., such as the mass storage device 1128 and/or the volatilememory 1114 included in the example processing system 1100 of FIG. 11,which is described in greater detail below. In such examples, theaudience recognizer 230 of FIG. 5 includes an example queue processor520 to assign key frames to the queue 515 with priorities that varydepending on whether the key image signatures representative of the keyframes match (or substantially match) at least one of the referencesignatures representative of the set of reference scenes depicting thereference audiences. For example, when assigning a particular key frameto the queue 515, the queue processor 520 can assign the key frame tothe queue 515 with a first priority when the signature comparator 505indicates that the key image signature corresponding to the key framedoes not match any of the reference signatures representative of thereference audiences. However, the queue processor 520 can assign the keyframe to the queue 515 with a second priority, which is lower than thefirst priority, when the signature comparator 505 indicates that the keyimage signature corresponding to the key frame matches at least one ofthe set of reference signatures representative of the referenceaudiences. In some examples, the queue processor 520 can assign keyframes to the queue 515 based on a range of different possiblepriorities corresponding to, for example, how closely a particular keyframe matches at least one of the reference images and/or how manyimages a key frame signature represents (e.g., corresponding to thenumber of images in the image sequence(s) represented by the key framesignature).

In such examples, the audience identifier 510 retrieves key frames fromthe queue 515 for audience recognition processing (e.g., such as facialrecognition processing) based on the priorities associated with therespective key frames. Accordingly, the priorities assigned to thedifferent key frames can be used to, for example, focus audiencerecognition processing on a new scene that does not match any of thereference scenes previously identified by the audience recognizer 230,or/and focus processing on those key frames that represent largernumbers of individual image frames. In some examples, because the keyframes may be processed by the audience identifier 510 out of order dueto their different priorities in the queue 515, the queue processor 520also associates a timestamp with a key frame when assigning it to thequeue to enable the resulting audience identification informationdetermined by the audience identifier 510 for the key frame to beassociated with the respective metering data determined by the sitemeter 125.

In some examples, the queue processor 520 also includes suggestioninformation for performing audience recognition processing on a scene'skey frame that is assigned to the queue 515. For example, if thesignature comparator 505 determines that the key image signaturerepresentative of the scene's key frame matches a reference signature,then the queue processor 520 can link suggestion information with thescene's key frame that includes likely facial positions, sizes,orientations, etc., that are associated with the identified audiencecorresponding to the matching reference signature. Such suggestioninformation can be used by the audience identifier 510 to, for example,initialize and/or guide a facial detection and recognition process withlikely facial positions, sizes, orientations, etc., to speed up suchprocessing.

After determining the audience identification information for theaudience depicted in a particular key frame (e.g., by perform audiencerecognition processing and/or by using audience identificationinformation already associated with a matching reference audience), theaudience identifier 510 reports the audience identification information,as described above. Additionally, in some examples, the audienceidentifier 510 stores the audience identification information along withthe key frame's image signature (and possibly the key frame, too) in thereference storage 235 for use as a reference image signature andassociated reference audience identification information when processinga subsequent key frame in the sequence of key frames representing thesequence of scenes depicting subsequent audience(s) 120 to beidentified.

While example manners of implementing the scene-based people meter 130of FIG. 1 has been illustrated in FIGS. 2-5, one or more of theelements, processes and/or devices illustrated in FIGS. 2-5 may becombined, divided, re-arranged, omitted, eliminated and/or implementedin any other way. Further, the example sensor interface 215, the examplescene change detector 225, the example audience recognizer 230, theexample signature generator 305, the example signature comparator 310,the example image segmenter 315, the example key frame identifier 320,the example image clusterer 325, the example signature comparator 505,the example audience identifier 510, the example queue 515, the examplequeue processor 520 and/or, more generally, the example scene-basedpeople meter 130 of FIGS. 1-5 may be implemented by hardware, software,firmware and/or any combination of hardware, software and/or firmware.Thus, for example, any of the example sensor interface 215, the examplescene change detector 225, the example audience recognizer 230, theexample signature generator 305, the example signature comparator 310,the example image segmenter 315, the example key frame identifier 320,the example image clusterer 325, the example signature comparator 505,the example audience identifier 510, the example queue 515, the examplequeue processor 520 and/or, more generally, the example scene-basedpeople meter 130 could be implemented by one or more circuit(s),programmable processor(s), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)), etc. When any of the apparatusor system claims of this patent are read to cover a purely softwareand/or firmware implementation, at least one of the example scene-basedpeople meter 130, the example sensor interface 215, the example scenechange detector 225, the example audience recognizer 230, the examplesignature generator 305, the example signature comparator 310, theexample image segmenter 315, the example key frame identifier 320, theexample image clusterer 325, the example signature comparator 505, theexample audience identifier 510, the example queue 515 and/or theexample queue processor 520 are hereby expressly defined to include atangible computer readable medium such as a memory, digital versatiledisk (DVD), compact disk (CD), Blu-ray disc™, etc., storing suchsoftware and/or firmware. Further still, the example scene-based peoplemeter 130 of FIGS. 1-5 may include one or more elements, processesand/or devices in addition to, or instead of, those illustrated in FIGS.1-5, and/or may include more than one of any or all of the illustratedelements, processes and devices.

Flowcharts representative of example machine readable instructions forimplementing the example scene-based people meter 130, the examplesensor interface 215, the example scene change detector 225, the exampleaudience recognizer 230, the example signature generator 305, theexample signature comparator 310, the example image segmenter 315, theexample key frame identifier 320, the example image clusterer 325, theexample signature comparator 505, the example audience identifier 510,the example queue 515 and/or the example queue processor 520 are shownin FIGS. 6-10. In these examples, the machine readable instructionsrepresented by each flowchart may comprise one or more programs forexecution by a processor, such as the processor 1112 shown in theexample processing system 1100 discussed below in connection with FIG.11. The one or more programs, or portion(s) thereof, may be embodied insoftware stored on a tangible computer readable medium such as a CD-ROM,a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-raydisc™, or a memory associated with the processor 1112, but the entireprogram or programs and/or portions thereof could alternatively beexecuted by a device other than the processor 1112 (e.g., such as acontroller and/or any other suitable device) and/or embodied in firmwareor dedicated hardware (e.g., implemented by an ASIC, a PLD, an FPLD,discrete logic, etc.). Also, one or more of the machine readableinstructions represented by the flowchart of FIGS. 6-10 may beimplemented manually. Further, although the example machine readableinstructions are described with reference to the flowcharts illustratedin FIGS. 6-10, many other methods of implementing the examplescene-based people meter 130, the example sensor interface 215, theexample scene change detector 225, the example audience recognizer 230,the example signature generator 305, the example signature comparator310, the example image segmenter 315, the example key frame identifier320, the example image clusterer 325, the example signature comparator505, the example audience identifier 510, the example queue 515 and/orthe example queue processor 520 may alternatively be used. For example,with reference to the flowcharts illustrated in FIGS. 6-10, the order ofexecution of the blocks may be changed, and/or some of the blocksdescribed may be changed, eliminated, combined and/or subdivided intomultiple blocks.

As mentioned above, the example processes of FIGS. 6-10 may beimplemented using coded instructions (e.g., computer readableinstructions) stored on a tangible computer readable medium such as ahard disk drive, a flash memory, a read-only memory (ROM), a CD, a DVD,a cache, a random-access memory (RAM) and/or any other storage media inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, brief instances, for temporarily buffering, and/orfor caching of the information). As used herein, the term tangiblecomputer readable medium is expressly defined to include any type ofcomputer readable storage and to exclude propagating signals.Additionally or alternatively, the example processes of FIGS. 6-10 maybe implemented using coded instructions (e.g., computer readableinstructions) stored on a non-transitory computer readable medium, suchas a flash memory, a ROM, a CD, a DVD, a cache, a random-access memory(RAM) and/or any other storage media in which information is stored forany duration (e.g., for extended time periods, permanently, briefinstances, for temporarily buffering, and/or for caching of theinformation). As used herein, the term non-transitory computer readablemedium is expressly defined to include any type of computer readablemedium and to exclude propagating signals. Also, as used herein, theterms “computer readable” and “machine readable” are consideredequivalent unless indicated otherwise. Furthermore, as used herein, whenthe phrase “at least” is used as the transition term in a preamble of aclaim, it is open-ended in the same manner as the term “comprising” isopen ended. Thus, a claim using “at least” as the transition term in itspreamble may include elements in addition to those expressly recited inthe claim.

Example machine readable instructions 600 that may be executed toimplement the example scene-based people meter 130 of FIGS. 1-5 arerepresented by the flowchart shown in FIG. 6. The example machinereadable instructions 600 may be executed at predetermined intervals,based on an occurrence of a predetermined event, etc., or anycombination thereof. With reference to the preceding figures, themachine readable instructions 600 of FIG. 6 begin execution at block 605at which the sensor interface 215 of the scene-based people meter 130causes the imaging sensor 205 to obtain (e.g., capture) image frames ofthe audience area 115 at a configured/specified frame rate. The sensorinterface 215 stores the obtained image frames in the image storage 220.As described above, the image frames obtained at block 605 depict alocation (e.g., the audience area 115) in which an audience (e.g., theaudience 120) is expected to be present.

At block 610, the scene change detector 225 of the scene-based peoplemeter 130 segments (groups) the image frames obtained by the imagesensor 205 and stored in the image storage 220 into a sequence of scenes(e.g., corresponding to a respective sequence of image frames) for whichaudience identification is to be performed. At block 610, the scenechange detector 225 also determines respective key frames to representthe detected scenes. As such, the key frame for a particular scenedepicts the composition of the audience 120 corresponding to that scene.As described above, each next scene in the sequence of scenes detectedby the scene change detector 225 at block 610 corresponds to a likelychange in the composition of the audience 120 in the audience area 115and/or in its environment.

At block 615, the audience recognizer 230 of the scene-based peoplemeter 130 performs audience recognition using the key frames thatrepresent the image sequences forming the respective scenes detected atblock 610. As described above, at block 615, the audience recognizer 230focuses audience recognition processing on just the key framesdetermined at block 610 instead of all of the image frames obtained atblock 605, which can significantly reduce the processing costsassociated with the scene-based people meter 130. Processing thenreturns to block 605 and blocks subsequent thereto to enable scene-basedpeople metering to continue at the monitored site 105. Although FIG. 6illustrates that processing at blocks 605-615 as being performedserially in a single thread, in some examples, the processing performedat some of all of blocks 605-615 can be performed at the same time(e.g., in parallel) in multiple (e.g., three) processing threads.

In some examples, at block 615 the audience recognizer 230 performsaudience recognition using just the representative key frames of thescene clusters formed by grouping similar scenes into clusters, asdescribed above. In such examples, the audience recognizer 230 performsaudience recognition on just the representative key frame for each scenecluster (e.g., and in an order based the ranking/priority of the keyframes representative of the clusters), and then propagates theresulting audience identification information determined forrepresentative key frame of a particular scene cluster to all key framesand corresponding scenes included in the particular scene cluster.

Example machine readable instructions 700 that may be executed toimplement the example scene change detector 225 of FIGS. 2-4 arerepresented by the flowchart shown in FIG. 7. The example machinereadable instructions 700 may be executed at predetermined intervals,based on an occurrence of a predetermined event, etc., or anycombination thereof. With reference to the preceding figures, themachine readable instructions 700 of FIG. 7 begin execution at block 705at which the signature generator 305 of the scene change detector 225obtains, from the image storage 220, a next image frame that depicts theaudience area 115 being monitored using the imaging sensor 205. At block710, the signature generator 305 generates an image signaturerepresentative of the image frame obtained at block 705, as describedabove. At block 715, the image segmenter 315 of the scene changedetector 225 uses the signature comparator 310 to compare the imagesignature generated at block 710 with the image signature(s) for theprior image frames already included in the image sequence for thecurrent scene being detected. For example, at block 715, the imagesegmenter 315 can use the signature comparator 310 to compare the imagesignature for the next image frame obtained at block 705 with the imagesignature for the starting frame of the sequence of images forming thecurrent scene being detected, as described above.

At block 720, the image segmenter 315 determines whether the imagesignature determined at block 710 for the next image frame obtained atblock 705 matches (or substantially matches) the image signature for theprior image frames already included in the image sequence forming thescene currently being detected (e.g., by determining whether the imagesignature for the next image frame matches the starting frame's imagesignature). If the signatures match (block 720), then at block 725 theimage segmenter 315 includes the next image frame obtained at block 705in the image sequence forming the current scene being detected, asdescribed above. However, if the signatures do not match (block 720),then at block 730 the image segmenter 315 concludes detection of thecurrent scene and uses the next image frame obtained at block 705 as thestarting frame for a new image sequence that will form the next scene tobe detected. Also, at block 735, the key frame identifier 320 of thescene change detector 225 determines key frame(s) to be used torepresent the image sequence for the current scene for which detectionhas concluded, as described above. Processing then returns to block 705and blocks subsequent thereto to enable a next scene to be detected andformed from the sequence of image frames depicting the audience area115.

First example machine readable instructions 800 that may be executed toimplement the example audience recognizer 230 of FIGS. 2 and 5 arerepresented by the flowchart shown in FIG. 8. The example machinereadable instructions 800 may be executed at predetermined intervals,based on an occurrence of a predetermined event, etc., or anycombination thereof. With reference to the preceding figures, themachine readable instructions 800 of FIG. 8 begin execution at block 805at which the signature comparator 505 of the audience recognizer 230obtains the key image signature representing the key frame of the nextscene for which audience recognition processing is to be performed. Atblock 810, the signature comparator 505 compares the key image signaturewith the set of reference signatures stored in the reference storage235, and which represent the set of reference scenes depicting the setof reference audiences for which the audience recognizer 230 hasperformed audience identification in the past, as described above.

At block 815, queue processor 520 examines the result of comparing thekey image signature with the set of reference signatures that wasdetermined by the signature comparator 505 at block 810. If thecomparison result indicates that the key image signature did not matchany of the reference signatures (block 815), then at block 820 the queueprocessor 520 assigns the key frame(s) obtained at block 805 to thequeue 515 with a first (e.g., high) priority, as described above. Theaudience identifier 510 will later retrieve the key frame from the queue515 for audience recognition processing in accordance with its assigned(e.g., higher) priority. However, if the comparison result indicatesthat the key image signature matched at least one of the referencesignatures (block 815), then at block 825 the queue processor 520assigns the key frame(s) obtained at block 805 to the queue 515 with asecond (e.g., low) priority, as described above. In some examples, thequeue processor 520 also includes suggestion information for performingaudience recognition processing on the key frame. For example, thesuggestion information can include likely facial positions, sizes,orientations, etc., that are associated with a matching referencesignature. Such suggestion information can be used to, for example,initialize and/or guide a facial detection and recognition process withlikely facial positions, sizes, orientations, etc., to speed up suchprocessing. The audience identifier 510 will later retrieve the keyframe (and any associated suggestion information) from the queue 515 foraudience recognition processing in accordance with its assigned (e.g.,lower) priority. Processing then returns to block 805 and blockssubsequent thereto to enable audience recognition to be performed for anext scene in the sequence of scenes depicting the audience area 115.

Second example machine readable instructions 900 that may be executed toimplement the example audience recognizer 230 of FIGS. 2 and 5 arerepresented by the flowchart shown in FIG. 9. The example machinereadable instructions 900 may be executed at predetermined intervals,based on an occurrence of a predetermined event, etc., or anycombination thereof. With reference to the preceding figures, themachine readable instructions 900 of FIG. 9 begin execution at block 905at which the audience identifier 510 of the audience recognizer 230retrieves a next key frame (and any associated audience recognitionsuggestion information) from the queue 515 for audience recognitionprocessing in accordance with the priorities assigned to the key framesin the queue 515, as described above. For example, the audienceidentifier 510 may retrieve key frames for audience recognitionprocessing in order of priority from highest to lowest.

At block 910, the audience identifier 510 determines whether audienceidentification information is already associated with the key framebeing processed. This may occur, for example, when the key frame wasdetermined to match a reference scene depicting a reference audience andthe audience identifier 510 has already associated the matchingreference audience's identification information with the current keyframe being processed. If audience identification information is notalready associated with the key frame (block 910), then at block 915 theaudience identifier 510 performs audience identification processing(e.g., such as facial recognition processing and/or any other type(s)and/or number of identification techniques) on the key frame retrievedfrom the queue 515 at block 905 (with the audience identificationprocessing being initialized with any suggestion information associatedwith the key frame, as described above). At block 915, the audienceidentifier 510 associates the identification information determined viathe recognition processing with the key frame being processed. However,if audience identification information is already associated with thekey frame (block 910), then at block 925 the audience identifier 510determines whether that pre-existing audience information (e.g., thereference audience information associated with the matching referencescene) is to be used to identify the audience depicted in the key frame,or if audience identification processing (e.g., such as facialrecognition processing) is to be performed on the key frame regardlessof any matching reference audience information that is present. Forexample, to prevent audience identification errors from propagatingindefinitely, the audience identifier 510 may select key frames havingmatching reference audience information at random (e.g., with someprobability) to undergo audience identification processing in lieu ofusing the reference audience identification information to identify theaudience in the key frame. If the audience identifier 510 determinesthat the matching reference audience information is to be used toidentify the audience depicted in the key frame (block 920), then atblock 925 the audience identifier 510 uses the matching referenceaudience identification information to identify the audience depicted inthe key frame. Otherwise, the audience identifier 510 identifies theaudience depicted in the key frame via the recognition processingperformed at block 910, as described above.

Next, at block 930, the audience identifier 510 stores the key frame'simage signature and the audience identification information determinedfor the key frame in the reference storage 235. This information thenbecomes further reference information for use in perform audiencerecognition processing on subsequent key frames. At block 935, thereports the audience identification information for the key frame beingprocessed to the site meter 125, as described above. Processing thenreturns to block 905 and blocks subsequent thereto to enable audiencerecognition to be performed for a next scene in the sequence of scenesdepicting the audience area 115.

Third example machine readable instructions 1000 that may be executed toimplement the example audience recognizer 230 of FIGS. 2 and 5 arerepresented by the flowchart shown in FIG. 10. The example machinereadable instructions 1000 may be executed at predetermined intervals,based on an occurrence of a predetermined event, etc., or anycombination thereof. With reference to the preceding figures, themachine readable instructions 1000 of FIG. 9 begin execution at block1005 at which the audience identifier 510 of the audience recognizer 230determines whether it is time to report people metering results to thedata processing facility 140. If it is time to report people meteringresults, then at block 1010 the audience identifier 510 reports theaudience identification information for the scenes for which processing(e.g., for which facial recognition processing, as described above) oftheir respective key frames has completed. At block 1015, the audienceidentifier 510 handles reporting of audience identification informationfor scenes for which key frame processing has not completed by the timethe people metering results are to be reported. In some examples, foreach scene for which key frame processing has not completed, at block1015 the audience identifier 510 uses the reference audienceidentification (e.g., reference demographic information) alreadyassociated with the reference audience corresponding to referencesignature that matched the key frame's image signature to identify theaudience 120 depicted in the key frame, as described above. In someexamples, if the key image signature of an unprocessed key frame did notmatch any reference signature, then at block 1015 the audienceidentifier 510 may report that the audience information for the scenecorresponding to that key frame is unknown or NULL.

FIG. 11 is a block diagram of an example processing system 1100 capableof executing the instructions of FIGS. 6-10 to implement the examplescene-based people meter 130, the example sensor interface 215, theexample scene change detector 225, the example audience recognizer 230,the example signature generator 305, the example signature comparator310, the example image segmenter 315, the example key frame identifier320, the example image clusterer 325, the example signature comparator505, the example audience identifier 510, the example queue 515 and/orthe example queue processor 520 of FIGS. 1-5. The processing system 1100can be, for example, a server, a personal computer, a mobile phone(e.g., a smartphone, a cell phone, etc.), a personal digital assistant(PDA), an Internet appliance, a DVD player, a CD player, a digital videorecorder, a Blu-ray player, a gaming console, a personal video recorder,a set top box, a digital camera, or any other type of computing device.

The system 1100 of the instant example includes a processor 1112. Forexample, the processor 1112 can be implemented by one or moremicroprocessors and/or controllers from any desired family ormanufacturer.

The processor 1112 includes a local memory 1113 (e.g., a cache) and isin communication with a main memory including a volatile memory 1114 anda non-volatile memory 1116 via a bus 1118. The volatile memory 1114 maybe implemented by Static Random Access Memory (SRAM), SynchronousDynamic Random Access Memory (SDRAM), Dynamic Random Access Memory(DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any othertype of random access memory device. The non-volatile memory 1116 may beimplemented by flash memory and/or any other desired type of memorydevice. Access to the main memory 1114, 1116 is controlled by a memorycontroller.

The processing system 1100 also includes an interface circuit 1120. Theinterface circuit 1120 may be implemented by any type of interfacestandard, such as an Ethernet interface, a universal serial bus (USB),and/or a PCI express interface.

One or more input devices 1122 are connected to the interface circuit1120. The input device(s) 1122 permit a user to enter data and commandsinto the processor 1112. The input device(s) can be implemented by, forexample, a keyboard, a mouse, a touchscreen, a track-pad, a trackball, atrackbar (such as an isopoint), a voice recognition system and/or anyother human-machine interface.

One or more output devices 1124 are also connected to the interfacecircuit 1120. The output devices 1124 can be implemented, for example,by display devices (e.g., a liquid crystal display, a cathode ray tubedisplay (CRT)), a printer and/or speakers. The interface circuit 1120,thus, typically includes a graphics driver card.

The interface circuit 1120 also includes a communication device, such asa modem or network interface card, to facilitate exchange of data withexternal computers via a network 1126 (e.g., an Ethernet connection, adigital subscriber line (DSL), a telephone line, coaxial cable, acellular telephone system, etc.).

The processing system 1100 also includes one or more mass storagedevices 1128 for storing machine readable instructions and data.Examples of such mass storage devices 1128 include floppy disk drives,hard drive disks, compact disk drives and digital versatile disk (DVD)drives. In some examples, the mass storage device 1030 may implement theimage storage 220, the reference storage 235 and/or the queue 515.Additionally or alternatively, in some examples the volatile memory 1118may implement the image storage 220, the reference storage 235 and/orthe queue 515.

Coded instructions 1132 corresponding to the instructions of FIGS. 6-10may be stored in the mass storage device 1128, in the volatile memory1114, in the non-volatile memory 1116, in the local memory 1113 and/oron a removable storage medium, such as a CD or DVD 1136.

As an alternative to implementing the methods and/or apparatus describedherein in a system such as the processing system of FIG. 11, the methodsand or apparatus described herein may be embedded in a structure such asa processor and/or an ASIC (application specific integrated circuit).

Finally, although certain example methods, apparatus and articles ofmanufacture have been described herein, the scope of coverage of thispatent is not limited thereto. On the contrary, this patent covers allmethods, apparatus and articles of manufacture fairly falling within thescope of the appended claims either literally or under the doctrine ofequivalents.

What is claimed is:
 1. A method to perform people metering for audiencemeasurement, the method comprising: segmenting image frames depicting alocation in which an audience is expected to be present to form asequence of scenes; for a first scene in the sequence of scenes,identifying a key frame representing a first sequence of image framescorresponding to the first scene; and processing the key frame toidentify an audience depicted in the first scene.
 2. A method as definedin claim 1 wherein segmenting the sequence of image frames comprises:determining image signatures representative of the image frames; andgrouping successive frames having similar image signatures in respectivesequences of the image frames, each sequence of image frames forming arespective scene in the sequence of scenes.
 3. A method as defined inclaim 1 wherein the key frame comprises at least one of a starting imageframe, a midpoint image frame, an ending video image or a statisticalcombination of the first sequence of image frames forming the firstscene.
 4. A method as defined in claim 1 wherein processing the keyframe comprises performing facial recognition processing on the keyframe to identify audience members depicted in the first scene.
 5. Amethod as defined in claim 1 wherein processing the key frame comprises:comparing an image signature representative of the key image with a setof reference signatures representative of a set of reference scenes todetermine a comparison result; and processing the key frame based on thecomparison result to identify audience members depicted in the firstscene.
 6. A method as defined in claim 5 wherein processing the keyframe based on the comparison result comprises: assigning the key frameto a processing queue with a first priority when the comparison resultindicates that the image signature does not match at least one referencesignature in the set of reference signatures; assigning the key frame tothe processing queue with a second priority when the comparison resultindicates that the image signature matches at least one referencesignature in the set of reference signatures, the second priority beinglower than the first priority; and using the priorities associated withkey frames included in the processing queue to prioritize performingfacial recognition processing on the key frames to identify the audiencemembers depicted in the respective scenes represented by the key frames.7. A method as defined in claim 5 wherein processing the key frame basedon the comparison result comprises, when the comparison result indicatesthat the image signature matches a first reference signature in the setof reference signatures, using audience identification informationassociated with a first reference scene represented by the firstreference signature to identify the audience members depicted in thefirst scene.
 8. A method as defined in claim 5 further comprising:associating audience identification information with the imagesignature; and including the image signature in the set of referencesignatures for comparison with a second image signature representativeof a subsequent scene occurring after the first scene.
 9. A tangiblemachine readable storage medium comprising machine readable instructionswhich, when executed, cause a machine to at least: segment image framesdepicting a location in which an audience is expected to be present toform a sequence of scenes; for a first scene in the sequence of scenes,identify a key frame representing a first sequence of image framescorresponding to the first scene; and process the key frame to identifyan audience depicted in the first scene.
 10. A storage medium as definedin claim 9 wherein to segment the sequence of image frames, the machinereadable instructions, when executed, further cause the machine to:determine image signatures representative of the image frames; and groupsuccessive frames having similar image signatures in respectivesequences of image frames, each sequence of image frames forming arespective scene in the sequence of scenes.
 11. A storage medium asdefined in claim 9 wherein the key frame comprises at least one of astarting image frame, a midpoint image frame, an ending video image or astatistical combination of the first sequence of image frames formingthe first scene.
 12. A storage medium as defined in claim 9 wherein toprocess the key frame, the machine readable instructions, when executed,further cause the machine to perform facial recognition processing onthe key frame to identify audience members depicted in the first scene13. A storage medium as defined in claim 9 wherein to process the keyframe, the machine readable instructions, when executed, further causethe machine to: compare an image signature representative of the keyimage with a set of reference signatures representative of a set ofreference scenes to determine a comparison result; and process the keyframe based on the comparison result to identify audience membersdepicted in the first scene.
 14. A storage medium as defined in claim 13wherein the machine readable instructions, when executed, further causethe machine to: assign the key frame to a processing queue with a firstpriority when the comparison result indicates that the image signaturedoes not match at least one reference signature in the set of referencesignatures; assign the key frame to the processing queue with a secondpriority when the comparison result indicates that the image signaturematches at least one reference signature in the set of referencesignatures, the second priority being lower than the first priority; anduse the priorities associated with key frames included in the processingqueue to prioritize performing facial recognition processing on the keyframes to identify the audience members depicted in the respectivescenes represented by the key frames.
 15. A storage medium as defined inclaim 13 wherein the machine readable instructions, when executed,further cause the machine to use audience identification informationassociated with a first reference scene represented by a first referencesignature to identify the audience members depicted in the first scenewhen the comparison result indicates that the image signature matchesthe first reference signature.
 16. A storage medium as defined in claim13 wherein the machine readable instructions, when executed, furthercause the machine to: associate audience identification information withthe image signature; and include the image signature in the set ofreference signatures for comparison with a second image signaturerepresentative of a subsequent scene occurring after the first scene.17. An apparatus to identify people in an audience, the apparatuscomprising: a scene change detector to segment image frames depicting alocation in which an audience is expected to be present to form asequence of scenes; and an audience recognizer to process a key framerepresenting a first sequence of image frames corresponding to a firstscene in the sequence of scenes to identify a person in an audiencedepicted in the first scene.
 18. An apparatus as defined in claim 17wherein the scene change detector is to: determine image signaturesrepresentative of the image frames; and include successive frames havingsimilar image signatures in respective sequences of image frames, eachsequence forming a respective scene in the sequence of scenes.
 19. Anapparatus as defined in claim 17 wherein the key frame comprises atleast one of a starting image frame, a midpoint image frame, an endingvideo image or a statistical combination of the first sequence of imageframes forming the first scene.
 20. An apparatus as defined in claim 17wherein the audience recognizer is to: compare an image signaturerepresentative of the key image with a set of reference signaturesrepresentative of a set of reference scenes to determine a comparisonresult; and process the key frame based on the comparison result toidentify the person depicted in the first scene.
 21. An apparatus asdefined in claim 20 wherein the audience recognizer is to: assign thekey frame to a processing queue with a first priority when thecomparison result indicates that the image signature does not match atleast one reference signature in the set of reference signatures; assignthe key frame to the processing queue with a second priority when thecomparison results indicates that the image signature matches at leastone reference signature in the set of reference signatures, the secondpriority being lower than the first priority; and use the prioritiesassociated with key frames included in the processing queue toprioritize performing facial recognition processing on the key frames toidentify the person depicted in the respective scenes represented by thekey frames.
 22. An apparatus as defined in claim 20 wherein the audiencerecognizer is to use first audience identification informationassociated with a first reference scene represented by a first referencesignature to identify the person depicted in the first scene when thecomparison result indicates that the image signature matches the firstreference signature.